Automatic trimming method, apparatus and program

ABSTRACT

Trimming is automatically performed based on a person or persons in whom a photographer has interest. All of facial images included in a whole image are detected. Then, judgment is made as to whether each of the detected facial images is a facial image of a specific person, face information about whom is stored in a face database. If the detected facial images include a facial image or images of the specific person or persons, trimming is performed based on the facial image or images of the specific person or persons.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an automatic trimming method andapparatus for automatically trimming, based on the face of a specificperson, a whole image obtained by a digital camera or the like. Thepresent invention also relates to a program for the automatic trimmingmethod and apparatus.

2. Description of the Related Art

A trimming method for producing a certificate photograph, which is usedto apply for a passport or a license or to prepare a resume or the like,is well known. In the trimming method, a face in an image is enlarged orreduced to satisfy a standard for the certificate photograph, andtrimming is performed so that the face is positioned at a predeterminedposition of the certificate photograph. Particularly, as techniques fordetecting a face, eyes or the like in a photograph image have beenimproved, an automatic trimming method has been proposed (for example,please refer to Japanese Unexamined Patent Publication No. 2005-242641).In the automatic trimming method, trimming is performed by detecting aface, an eye or the like in a photograph image and by setting a trimmingarea based on the detection result. In Japanese Unexamined PatentPublication No. 2005-242641, first, a face, an eye or eyes and a pupilor pupils are detected in a photograph image. Then, the positions of thepupils and a distance between the pupils are calculated. Further, atrimming frame is set, based on the positions of the pupils and thedistance therebetween, so that the whole face is included in thetrimming frame. After then, data for trimming is produced by attachinginformation about the trimming frame to the photograph image.

Besides the aforementioned automatic trimming method for a certificatephotograph, a method for automatically performing trimming on an imageobtained by a user using a digital camera or the like has been proposed(for example, please refer to Japanese Unexamined Patent Publication No.2005-267454). In Japanese Unexamined Patent Publication No. 2005-267454,first, human faces are detected in a photograph image. Then, theposition and the size of a trimming area are set based on the positionsof all of the detected faces so that all of the faces are included inthe trimming area. Automatic trimming is performed based on the trimmingarea.

In Japanese Unexamined Patent Publication No. 2005-267454, the trimmingframe is set so that all of the detected faces are included in thetrimming frame. Therefore, a person such as a passerby for example, whodoes not need to be included in the trimming frame, is included therein.Consequently, when an image is obtained by trimming, a person in whom aphotographer has interest is positioned close to an edge of the image orthe size of the person becomes too small. Hence, there is a problem thatautomatic trimming is not performed based on the images of persons whichthe photographer wants to obtain.

SUMMARY OF THE INVENTION

In view of the foregoing circumstances, it is an object of the presentinvention to provide an automatic trimming method and apparatus forautomatically performing trimming based on a specific person or personsin whom a photographer has interest. It is also an object of the presentinvention to provide a program for the automatic trimming method andapparatus.

An automatic trimming method of the present invention is an automatictrimming method comprising the steps of:

detecting a facial image in a whole image;

recognizing whether the detected facial image is a facial image of aspecific person, face information about whom is registered in a facedatabase; and

if the detected facial image is recognized as a facial image of thespecific person, the face information about whom is registered in theface database, automatically performing trimming based on the positionof the recognized facial image.

An automatic trimming apparatus of the present invention is an automatictrimming apparatus comprising:

a face detection means for detecting a facial image in a whole image;

a face database in which face information about a specific person isregistered;

a recognition means for recognizing whether the facial image detected bythe face detection means is a facial image of the specific person, theface information about whom is registered in the face database; and

a trimming means, wherein if the recognition means recognizes that thedetected facial image is a facial image of the specific person, the faceinformation about whom is registered in the face database, the trimmingmeans automatically performs trimming based on the position of therecognized facial image.

An automatic trimming program of the present invention is an automatictrimming program for causing a computer to execute an automatic trimmingmethod, the program comprising the procedures for:

detecting a facial image in a whole image;

recognizing whether the detected facial image is a facial image of aspecific person, face information about whom is registered in a facedatabase; and

if the detected facial image is recognized as a facial image of thespecific person, the face information about whom is registered in theface database, automatically performing trimming based on the positionof the recognized facial image.

Here, the trimming means may automatically perform trimming by using anykind of method as long as automatic trimming is performed based on theposition of the facial image of the person, the face information aboutwhom is registered. For example, the trimming means may automaticallyperform trimming so that the facial image is positioned at apredetermined position of a trimming frame.

The face database may be a database in which a single facial image isregistered. Alternatively, the face database may be a database in whicha plurality of facial images are registered. If the recognition meansrecognizes that a plurality of facial images of specific persons, faceinformation about whom is registered in the face database, are presentin the whole image, the trimming means may set a trimming frame for eachof the plurality of facial images. Alternatively, the trimming means mayset a trimming frame so that the plurality of recognized facial imagesare included in a trimming frame.

The face detection means may use any kind of face detection method. Forexample, the face detection means may include a partial image productionmeans for producing a plurality of partial images by scanning the wholeimage using a subwindow formed by a frame of a set number of pixels anda face classifier for detecting a facial image included in the pluralityof partial images produced by the partial image production means.Further, the face classifier may judge whether each of the plurality ofpartial images is a facial image using a plurality of classificationresults obtained by a plurality of weak classifiers.

Further, the face information may be a facial image. Alternatively, theface information may be a feature value of a face.

In the automatic trimming method, apparatus and program of the presentinvention, a facial image is detected in a whole image, and judgment ismade as to whether the detected facial image is a facial image of aspecific person, face information about whom is registered in a facedatabase. If the detected facial image is recognized as a facial imageof the specific person, the face information about whom is registered inthe face database, trimming is automatically performed based on theposition of the recognized facial image. Therefore, even if the face ofa person such as a passerby, who has no relationship with thephotographer, is present in the whole image, it is possible to set atrimming frame based on the specific person in whom the photographer hasinterest. Hence, it is possible to automatically perform trimming sothat the intention of the photographer is reflected in an image obtainedby trimming.

If the trimming means performs trimming on the whole image so that thefacial image is positioned at a predetermined position of a trimmingframe, it is possible to automatically produce a trimming image that hasdesirable composition.

Further, if the recognition means recognizes that a plurality of facialimages of specific persons, face information about whom is registered inthe face database, are present in the whole image, the trimming meansmay set a trimming frame so that the plurality of recognized facialimages are included in the trimming frame. If the trimming means setsthe trimming frame in such a manner, a plurality of persons that arepresent in the whole image, and in whom the photographer has interest,can be surely included in the trimming frame.

Further, the face detection means may include a partial image productionmeans for producing a plurality of partial images by scanning the wholeimage using a subwindow formed by a frame of a set number of pixels anda face classifier for detecting a facial image included in the pluralityof partial images produced by the partial image production means.Further, the face classifier may judge whether each of the plurality ofpartial images is a facial image using a plurality of classificationresults obtained by a plurality of weak classifiers. If the facedetection means detects a face or faces in such a manner, it is possibleto detect faces accurately and efficiently.

Note that the program of the present invention may be provided beingrecorded on a computer readable medium. Those who are skilled in the artwould know that computer readable media are not limited to any specifictype of device, and include, but are not limited to: floppy disks, CD'sRAM's, ROM's, hard disks, magnetic tapes, and internet downloads, inwhich computer instructions can be stored and/or transmitted.Transmission of the computer instructions through a network or throughwireless transmission means is also within the scope of this invention.Additionally, computer instructions include, but are not limited to:source, object and executable code, and can be in any language includinghigher level languages, assembly language, and machine language.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an automatic trimming apparatusin a preferred embodiment of the present invention;

FIG. 2 is a block diagram illustrating an example of a face detectionmeans in the automatic trimming apparatus illustrated in FIG. 1;

FIG. 3A is a schematic diagram illustrating how a partial imageproduction means, illustrated in FIG. 2, scans a whole image using asubwindow;

FIG. 3B is a schematic diagram illustrating how the partial imageproduction means, illustrated in FIG. 2, scans a whole image using asubwindow;

FIG. 3C is a schematic diagram illustrating how the partial imageproduction means, illustrated in FIG. 2, scans a whole image using asubwindow;

FIG. 3D is a schematic diagram illustrating how the partial imageproduction means, illustrated in FIG. 2, scans a whole image using asubwindow;

FIG. 4A is a schematic diagram illustrating examples of front-view facesdetected by a face detection means, illustrated in FIG. 2;

FIG. 4B is a schematic diagram illustrating examples of profile facesdetected by the face detection means, illustrated in FIG. 2;

FIG. 5 is a schematic diagram illustrating how weak classifiersillustrated in FIG. 2 extract feature values from partial images;

FIG. 6 is a graph as an example of a histogram included in each of theweak classifiers illustrated in FIG. 2;

FIG. 7 is a schematic diagram illustrating an example of a whole imagein which a plurality of facial images are detected;

FIG. 8 is a schematic diagram illustrating an example of an imageobtained by automatically performing trimming on the whole imageillustrated in FIG. 7; and

FIG. 9 is a flowchart illustrating an automatic trimming method in apreferred embodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, preferred embodiments of an automatic trimming apparatus ofthe present invention will be described in detail with reference to theattached drawings. The configuration of an automatic trimming apparatus1, illustrated in FIG. 1, is realized by causing a computer (forexample, a personal computer or the like) to execute an automatictrimming program that has been read in an auxiliary storage device. Inthat case, the automatic trimming program is stored in an informationstorage medium, such as a CD-ROM, or distributed through a network, suchas the Internet. Then, the automatic trimming program is installed inthe computer. Alternatively, the automatic trimming program may beinstalled in a processor of a digital camera to realize theconfiguration of the automatic trimming apparatus 1.

The automatic trimming apparatus 1 automatically performs trimming bysetting a trimming frame in a whole image P obtained by a digital cameraor the like. The automatic trimming apparatus 1 includes a facedetection means 10, a face database 20, a recognition means 30 and atrimming means 40. The face detection means 10 detects a facial image Fin the whole image P. In the face database 20, a specific person orpersons are registered. The recognition means 30 recognizes whether thefacial image F is a facial image of a specific person registered in theface database 20. The trimming means 40 automatically performs trimmingbased on the position of the facial image F if the recognition means 30judges that the facial image F is a facial image of the specific personregistered in the face database 20.

The face detection means 10 includes a partial image production means11, a front-view face classification means 12A and a profile faceclassification means 12B, as illustrated in FIG. 2. The partial imageproduction means 11 produces a plurality of partial images PP byscanning the whole image P using a subwindow W. The front-view faceclassification means 12A detects a partial image or images, eachrepresenting a front-view face, from the plurality of partial images PP,produced by the partial image production means 11. The profile faceclassification means 12B detects a partial image or images, eachrepresenting a profile face (side-view face), from the plurality ofpartial images PP, produced by the partial image production means 11.

Before the whole image P is input to the partial image production means11, a preprocessing means 10 a performs preprocessing on the whole imageP. The preprocessing means 10 a has a function for producing a pluralityof whole images at multiple resolution levels from the whole image P.The preprocessing means 10 a produces whole images P2, P3 and P4, whichhave different resolution levels from each other, as illustrated inFIGS. 3A through 3D. Further, the preprocessing means 10 a has afunction for normalizing (hereinafter, referred to as localnormalization) contrast in the entire area of the whole image P so thatthe contrast in the entire area thereof becomes a predetermined level.The preprocessing means 10 a normalizes the contrast in the entire areaof the whole image P by reducing the variation of contrast in a localarea thereof.

The partial image production means 11 scans the whole image P using asubwindow W including a set number of pixels (for example, 32 pixels×32pixels) and extracts an area surrounded by the subwindow W, asillustrated in FIG. 3A. Accordingly, the partial image production means11 produces partial images PP, each including the set number of pixels.Particularly, the partial image production means 11 produces the partialimages PP by moving the subwindow W by a constant number of pixels eachtime.

The partial image production means 11 also produces partial images PP byscanning each of produced low-resolution images using a subwindow W, asillustrated in FIGS. 3B through 3D. Since the partial image productionmeans 11 also produces the partial images PP from the low-resolutionimages, even if a face in the whole image P is too large to be includedin the subwindow W, the same face in a low-resolution image can beincluded in the subwindow W. Therefore, it is possible to surely detectfaces.

The front-view face classification means 12A and the profile faceclassification means 12 detect facial images F, for example, by usingAdaboosting algorithm. The front-view face classification means 12A hasa function for detecting in-plane-rotated front-view faces (please referto FIG. 4A). The front-view face classification means 12A includes 12front-view face classifiers 13-1 through 13-12. The rotation angles ofthe front-view face classifiers are different from each other by 30degrees within the range of 30 to 330 degrees. Further, each of thefront-view face classifiers 13-1 through 13-12 can detect faces atrotation angles of −15 (=345 degrees) through +15 degrees with respectto zero degree. The profile face classification means 12B has a functionfor detecting in-plane-rotated profile faces (please refer to FIG. 4B).For example, the profile face classification means 12B includes 7profile face classifiers 14-1 through 14-7, of which the rotation anglesare different from each other by 30 degrees within the range of −90 to+90 degrees. Further, the profile face classification means 12B mayinclude a profile face classifier for detecting out-of-plane-rotatedfacial images. The out-of-plane-rotated facial images are images, eachincluding a face, of which the direction has been further rotated(out-of-plane rotation) from that of an in-plane-rotated face.

Each of the front-view face classifiers 13-1 through 13-12 and theprofile face classifiers 14-1 through 14-7 has a function for judgingwhether a partial image PP is a facial image or a non-facial image usingtwo values. Further, each of the front-view face classifiers 13-1through 13-12 and the profile face classifiers 14-1 through 14-7includes a plurality of weak classifiers CF₁ through CF_(M) (M: thenumber of weak classifiers). Each of the weak classifiers CF₁ throughCF_(M) has a function for judging whether a partial image PP is a facialimage or not using a feature value x by extracting the feature value xfrom the partial image PP. Finally, each of the face classifier means12A and 12B judges whether the partial image PP is a facial image or notby using judgment results obtained by the weak classifiers CF₁ throughCF_(M).

Specifically, each of the weak classifiers CF₁ through CF_(M) extractsbrightness values or the like at coordinates P1 a, P1 b and P1 c, whichare set in the partial image PP (PP1), as illustrated in FIG. 5.Further, each of the weak classifiers CF₁ through CF_(M) extractsbrightness values or the like at coordinate positions P2 a and P2 b,which are set in a low-resolution image PP2 of the partial image PP.Further, each of the weak classifiers CF₁ through CF_(M) extractsbrightness values or the like at coordinate positions P3 a and P3 b,which are set in a low-resolution image PP3 of the partial image PP.Then, a difference in brightness between two of the seven coordinates P1a through P3 b is used as a feature value x. Each of the weakclassifiers CF₁ through CF_(M) uses a different feature value from eachother. For example, the weak classifier CF₁ uses a difference inbrightness between coordinate P1 a and coordinate P1 c as a featurevalue, and the weak classifier CF₂ uses a difference in brightnessbetween coordinate P2 a and coordinate P2 b as a feature value.

In the above description, a case in which each of the weak classifiersCF₁ through CF_(M) extracts a feature value x has been described as anexample. Alternatively, the aforementioned feature values x may beextracted in advance for a plurality of partial images PP and theextracted feature values x may be input to the weak classifiers CF₁through CF_(M), respectively. In the above example, the brightness valuewas used to obtain the feature value. Alternatively, other information,such as contrast and edge, may be used to obtain the feature value.

Each of the weak classifiers CF₁ through CF_(M) includes a histogram asillustrated in FIG. 6. The weak classifiers CF₁ through CF_(M) output,based on the histograms, scores f₁(x) through f_(M)(x) corresponding torespective feature values x. Further, the weak classifiers CF₁ throughCF_(M) include confidence levels β₁ through β_(M), each representing thejudgment performance thereof. The weak classifiers CF₁ through CF_(M)calculate judgment scores β_(m)·f_(m)(x) using scores f₁(x) throughf_(M)(x) and confidence levels β₁ through β_(M). Then, judgment is madeas to whether the value of the judgment score β_(m)·f_(m)(x) of each ofthe weak classifiers CF_(m) is greater than or equal to a set thresholdvalue Sref. If the value of the judgment score β_(m)·f_(m)(x) is greaterthan or equal to the set threshold value Sref, the partial image isrecognized as a facial image (β_(m)·f_(m)(x)≧Sref).

Further, each of the weak classifiers CF₁ through CF_(M) has a cascadestructure. Only when all of the week classifiers CF₁ through CF_(M) havejudged that a partial image PP is a facial image, the partial image PPis output as a facial image F. Specifically, weak classifier CF_(m+1) onthe downstream side of a weak classifier CF_(m) performs judgment onlyon a partial image PP that has been judged as a facial image by the weakclassifier CF_(m). In other words, if a partial image PP is judged as anon-facial image by the weak classifier CF_(m), the weak classifierCF_(m+1) on the downstream side of the weak classifier CF_(m) does notperform judgment on the partial image PP. Accordingly, it is possible toreduce the number (data amount) of partial images PP that should bejudged by weak classifiers on the downstream side. Hence, it is possibleto increase the speed of judgment operation. The detail of a classifierthat has a cascade structure is disclosed in Shihong LAO et al., “FastOmni-Direction Face Detection”, Meeting on Image Recognition andUnderstanding (MIRU2004), pp. II271-II276, 2004.

Further, each of the classifiers 13-1 through 13-12 and 14-1 through14-7 includes weak classifiers that have learned front-view faces orprofile faces as correct-answer sample images. The front-view faces andthe profile faces that have been learned by the weak classifiers arein-plane-rotated faces that should be judged by the respective weakclassifiers, and which are rotated at predetermined angles. Further, itis not necessary to separately judge whether each of the judgment scoresS₁ through S_(M), which have been output from the weak classifiers CF₁through CF_(M), is greater than or equal to a judgment score thresholdvalue Sref. Alternatively, the weak classifier CFm may judge whether thesum (Σ_(r=1) ^(m)βr·fr) of the judgment scores by weak classifiers CF₁through CF_(m−1) on the upstream side of the weak classifier CF_(m) isgreater than or equal to a judgment score threshold value S1ref (Σ_(r=1)^(m)βr·fr(x)≧S1ref). Accordingly, the judgment scores by theupstream-side weak classifiers can be taken into consideration to makejudgment. Therefore, it is possible to improve the accuracy of judgment.

Further, as an example of the face detection means 10, a case in whichfaces are detected by using Adaboosting algorithm has been described.Alternatively, faces may be detected by using SVM (Support VectorMachine) algorithm or other known face detection algorithm, such as aface detection method disclosed in Ming-Hsuan Yang et al., “DetectingFaces in Images: a Survey”, IEEE Transactions on Pattern Analysis andMachine Intelligence, vol. 24, No. 1, pp. 34-58, 2002.

In the face database 20, illustrated in FIG. 1, information about afacial image F of a specific person in whom a photographer has interestis registered in advance. The specific person in whom the photographerhas interest is a person, such as the photographer, him/herself, or aperson with whom the photographer has a relationship. For example, thefacial image in the face database 20 is an image registered in the pastby using an image including the photographer or the like. Therecognition means 30 recognizes whether a facial image F is a facialimage of a specific person by judging whether the facial image F is afacial image registered in the face database 20. Here, faces may berecognized by using various kinds of methods. For example, well-knownface recognition techniques, such as a method for recognizing a face byusing a local feature value of a facial image, a method for recognizinga face by using Adaboosting algorithm, a graph matching method and atemplate matching method, may be used. The face recognition techniquesare described in detail in W. Zhao et al., “Face Recognition: ALiterature Survey”, ACM Computing Surveys, Vol. 35, No. 4, pp. 399-458,2003, S. Lao et al., “a Survey on Face Detection and Face Recognition”,CVIM, May 2005 (149th) Meeting, H. Sakano, “Principal Component Analysisin Pattern Recognition—From the Viewpoint of Facial Image Recognition—”,Proceedings of the Institute of Statistical Mathematics, vol. 49, No. 1,pp. 23-42, 2001, and the like. Meanwhile, information registered in theface database 20 is face information (for example, a local feature valueof a facial image, a facial image, itself, or the like) in a format thatis most appropriate for the aforementioned face recognition algorithm.

The trimming means 40 has a function for performing trimming so that afacial image F recognized by the recognition means 30 is positioned at apredetermined position of a trimming frame. For example, if the facedetections means 10 detects four facial images F1 through F4 in a wholeimage P illustrated in FIG. 7 and the recognition means 30 recognizestwo facial images F1 and F2, the trimming means 40 sets a trimming frameTG as illustrated in FIG. 8. The trimming means 40 sets the trimmingframe so that the facial image F is positioned at a position satisfyingthe ratio of 1 (up):1.618 with respect to the vertical direction of thetrimming frame TG. At the same time, the trimming means 40 automaticallysets the trimming frame TG with respect to the horizontal direction sothat the midpoint of an area including the two facial image F1 and F2with respect to the horizontal direction becomes the same as the centerof the trimming frame TG. Since the position of the trimming frame TG isdetermined based on the aforementioned ratio, it is possible toautomatically produce a trimming image that has desirable composition.

As described above, trimming is automatically performed based on thefacial images F1 and F2 recognized by the recognition means 30.Therefore, it is possible to perform trimming based on a person orpersons in whom the photographer has interest. Specifically, if atrimming frame TG is set based on all of the detected facial images F1through F4, as in the conventional method, the facial images F1 and F2,in whom the photographer has interest, are positioned off the center ofthe trimming frame TG. Further, the sizes of the facial images F1 and F2become relatively small. Hence, it is difficult to perform trimming asdesired by the photographer. In contrast, if trimming is automaticallyperformed based on the facial images F1 and F2, recognized by therecognition means 30, as illustrated in FIG. 8, even if facial images F3and F4, such as facial images of passersby, in whom the photographerdoes not have interest, are detected, it is still possible to set atrimming frame TG based on the facial images F1 and F2 of persons inwhom the photographer has interest. Consequently, it is possible toautomatically perform trimming so that the intention of the photographeris reflected in an image obtained by trimming.

Further, as illustrated in FIG. 8, if the recognition means 30recognizes that the plurality of facial images F1 and F2 are facialimages of persons, the persons being registered in the face database 20,the trimming means 40 sets a trimming frame TG so that the plurality offacial images F1 and F2 are included in the trimming frame TG.Therefore, when a plurality of persons in whom the photographer hasinterest are present in a whole image P, it is possible to surelyinclude the plurality of persons in the trimming frame.

FIG. 9 is a flowchart showing a preferred embodiment of an automatictrimming method according to the present invention. The automatictrimming method will be described with reference to FIGS. 1 through 9.First, the face detection means 10 detects a facial image F in a wholeimage P (step ST1). Then, the recognition means 30 recognizes whetherthe detected face is the face of a specific person, the specific personbeing stored in the face database 20 (step ST2). If the face of thespecific person is detected, the trimming means 40 performs trimmingbased on the face of the specific person (step ST3).

In the aforementioned embodiments, a facial image F is detected in awhole image P, and processing is performed to recognize whether thedetected facial image F is a facial image of a specific person, thespecific person being registered in the face database 20. If it isrecognized that the facial image F is a facial image of the specificperson, trimming is automatically performed based on the position of therecognized facial image F. Therefore, even if the face of a person, suchas a passerby, who has no relationship with the photographer is presentin the whole image P, it is possible to set a trimming frame based onthe specific person in whom the photographer has interest. Hence, it ispossible to automatically perform trimming so that the intention of thephotographer is reflected in an image obtained by trimming.

The embodiments of the present invention are not limited to theaforementioned embodiments. For example, in FIG. 8, when a plurality ofdetected faces are faces registered in the database 20, the trimmingframe TG is set so that the plurality of facial images F are included inthe trimming frame TG. However, the trimming means 40 may set aplurality of trimming frames based on respective facial images. Further,the plurality of trimming frames may be displayed on a display unit sothat the photographer can select an appropriate trimming frame from theplurality of trimming frames.

Further, in the aforementioned embodiments, a case in which the trimmingmeans 40 automatically sets a trimming frame has been described as anexample. In addition, a correction means may be provided so that a usercan correct the position and the size of the trimming frame TG after thetrimming frame is automatically set.

Further, the automatic trimming apparatus in the aforementionedembodiments may be mounted on a photography apparatus, such as a digitalcamera. Further, when a whole image P is obtained by remote photographyusing a remote-operation camera or the like, the direction, the zoom,the focus or the like of the remote-operation camera may be controlledbased on composition obtained by performing the aforementioned automatictrimming.

Further, when the trimming means 40 sets a trimming frame TG based onthe facial image F, a standard position and a standard size of a personarea may be set in advance for each photography mode, such as a portraitmode and a person photography mode. Then, a trimming frame TG may be setso that the position and the size of a person area become appropriatefor the photography mode of photography. For example, if a whole image Pis obtained by photography in a portrait mode, the trimming means 40sets a trimming frame TG so that the size of a person area becomeslarger than that of a person area obtained in a mode other than theportrait mode.

Further, when a trimming frame TG is set, the position and the size ofthe trimming frame TG may be corrected by using information other thaninformation about the person in combination with the aforementionedinformation. For example, as disclosed in Japanese Unexamined PatentPublication No. 2004-310753, the meaning of an object area may beidentified with reference to the whole image P, and a trimming frame TGmay be set so that a specific object area (for example, structures,buildings or the like) is included in the trimming frame TG. When thetrimming frame TG is set in such a manner, first, the trimming frame TGmay be set so that the position and the size of a person area become apredetermined position and a predetermined size. Then, the position andthe size of the person area may be corrected so that the object area isincluded in an image obtained by trimming.

In the aforementioned example, a case in which the recognition means 30recognizes a specific person by face recognition (authentication) hasbeen described. Alternatively, a subject of photography may beidentified by receiving a signal from an IC (integrated circuit) tagcarried by the subject of photography. Further, when a trimming frame TGis set, a photography position and a photography direction may beobtained from an IC tag carried by a subject of photography (person) orGPS (global positioning system) and a direction sensor, as disclosed inJapanese Unexamined Patent Publication No. 2002-10114. Then, thetrimming frame TG may be set based on conditions estimated from thephotography position and the photography direction (for example, byobtaining information about scenery or structures in the surroundings ofthe place of photography from a database, such as a map informationdatabase). For example, the trimming means 40 may store the position andthe size of a person area in advance for each photography position andfor each photography direction. Then, when a photography position and aphotography direction are obtained from the tag information or the likeof the whole image P, a trimming frame TG may be set so that theposition and the size of a person area become the stored position andthe stored size.

1. An automatic trimming method comprising the steps of: detecting afacial image in a whole image; recognizing whether the detected facialimage is a facial image of a specific person, face information aboutwhom is registered in a face database; and if the detected facial imageis recognized as a facial image of the specific person, the faceinformation about whom is registered in the face database, automaticallyperforming trimming based on the position of the recognized facialimage.
 2. An automatic trimming apparatus comprising: a face detectionmeans for detecting a facial image in a whole image; a face database inwhich face information about a specific person is registered; arecognition means for recognizing whether the facial image detected bythe face detection means is a facial image of the specific person, theface information about whom is registered in the face database; and atrimming means, wherein if the recognition means recognizes that thedetected facial image is a facial image of the specific person, the faceinformation about whom is registered in the face database, the trimmingmeans automatically performs trimming based on the position of therecognized facial image.
 3. An automatic trimming apparatus, as definedin claim 2, wherein the trimming means performs trimming on the wholeimage so that the recognized facial image is positioned at apredetermined position of a trimming frame.
 4. An automatic trimmingapparatus, as defined in claim 2, wherein if the recognition meansrecognizes that a plurality of facial images of specific persons, faceinformation about whom is registered in the face database, are presentin the whole image, the trimming means sets a trimming frame so that theplurality of facial images are included in the trimming frame.
 5. Anautomatic trimming apparatus, as defined in claim 2, wherein the facedetection means includes a partial image production means for producinga plurality of partial images by scanning the whole image using asubwindow formed by a frame of a set number of pixels and a faceclassifier for detecting a facial image included in the plurality ofpartial images produced by the partial image production means, andwherein the face classifier judges whether each of the plurality ofpartial images is a facial image using a plurality of classificationresults obtained by a plurality of weak classifiers.
 6. An automatictrimming apparatus, as defined in claim 2, wherein the face informationis a facial image.
 7. An automatic trimming apparatus, as defined inclaim 2, wherein the face information is a feature value of a face.
 8. Acomputer readable storage medium storing an automatic trimming programfor causing a computer to execute an automatic trimming method, theprogram comprising the procedures for: detecting a facial image in awhole image; recognizing whether the detected facial image is a facialimage of a specific person, face information about whom is registered ina face database; and if the detected facial image is recognized as afacial image of the specific person, the face information about whom isregistered in the face database, automatically performing trimming basedon the position of the recognized facial image.